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Abstract 

Applying the Tubular Neighborhood Theorem, we give a short and new proof of the 
Pontryagin Maximum Principle on a smooth manifold. The idea is as follows. Given 
a control system on a manifold M, we embed it into an open subset of some R", and 
extend the control system to the open set. Then, we apply the Pontryagin Maximum 
Principle on R" to the extended system and project the consequence to M. 

keywords : optimal control, Pontryagin maximum principle, tubular neighborhood 

1 Introduction 

The classic book by Pontryagin et aL[S] gives a proof of the celebrated Pontryagin Maximum 
Principle (PMP) for control systems on R". See also [5] for another proof of the PMP in 
M". Although several books or journal articles have mentioned versions of the PMP for 
control systems on a smooth manifold, its proofs began to appear in the literature quite 
recently [H [3]. In general there can be three kinds of proof of the PMP on manifolds. 
The first is to translate the proof in |B] into the modern differential-geometric language, 
Although this approach gives a good geometric insight into the principle, it has the 
drawback that the proof becomes long since it follows the original proof in [5] . The second 
kind of the proof is to adapt the proof in [8] to manifolds by patching up a finite number of 
local charts covering an optimal trajectory without use of any modern differential-geometric 
machinery. A drawback of the second approach is that the proof becomes very long too, 
involving coordinate transformations and repeating the proof in [H]. To our knowledge, 
there is no literature containing the second kind of proof. The third kind of proof is the one 
that we present in this paper, which is a new and short proof of the PMP on manifolds, by 
combining the Tubular Neighborhood Theorem and the PMP on M". This proof does not 
repeat the proof in [8] , and is thus much shorter than and different from the two kinds of 
proofs mentioned above. 

The idea in our proof is simple. Given a control system on a manifold M , we embed M 
into some R", take a tubular neighborhood V of M in R", and construct a control system 
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on V whose restriction to Af agrees with the original system. Since M is an invariant 
manifold for the extended system on R", we can reformulate the original optimal control 
problem with a point-to-point transfer on M into an equivalent optimal control problem 
with a point-to-submanifold transfer in R" where the submanifold is transversal to M. We 
apply the PMP on R" to the equivalent problem, and then project (or, restrict) the result 
to M, to prove the PMP on M. Our proof is pedagogically meaningful in optimal control 
theory, and it illustrates a nice application of the Tubular Neighborhood Theorem to control 
theory. 

2 Main Results 

2.1 Review of the Pontryagin Maximum Principle on M"^ 

Consider a control system on R": 

i = x£W,ueW. (1) 

We want to find an optimal control u{t) such that 

/ /''(x, u)c!t is minimized (2) 

J to 

with free terminal time ti, and 

x{to) = xq, x{ti) = Xi. (3) 

For convenience, we assume that is a subset of a Euclidean space and that / : R" xW ^ 
R" and /° : R" X ^ R are smooth. 

Theorem 2.1 ([HI [S]). Suppose that u{t),tQ < t < ti is a piecewise continuous optimal 
control and x{t) is the corresponding optimal trajectory for flp - Then, there exists a 
non-vanishing continuous curve {pQ{t),p(t)) G R x R" = R x r*jj^R" such that: 

1. The trajectory {x{t),p{t)) satisfies 

dH . dH . ^ 
X = Pi = -^, « = l,...,n, (4) 

opi ox' 

i.e., it is a flow of the Hamiltonian vector field 

XH{x,p-po,u{t)) = n^dH (5) 

where f2 is the canonical symplectic form on r*R" = R" x R" and the Hamiltonian 
H is given by 

H{x,p,pq,u) ^pQf"{x,u) + {p,f{x,u)). 

2. u{t) — a,rgma,Xy(zw H{x{t),p{t),po{t),v) for every t G [to,ti]. 

3. Po <i and is constant in t. 

4. H{x{t),p{t),po,u{t)) ^ for every t e [tQ,ti]. 
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Theorem 2.2 ([HIIS])- Let u(t),to <t<ti, be a piecewise continuous optimal control and 
x{t) the corresponding trajectory for (QP, and 

x{to) £ So, x{ti) e Si 

where So and Si are smooth submanifolds o/M". Then, all of the conclusions in Theorem \2.1\ 
hold, and additionally the transversality conditions 

{p{to), T^(to)So) = 0, {p{ti), T^(^ti)Si) = 

are satisfied. 

Remark. Theorems 12.11 and 12.21 will still hold for a control system on an open subset U 
of R" if every trajectory of the control system leaves U (positively) invariant. Since the 
cotangent lift of a diffeomorphism of U onto a set F is a symplectomorphism of {T*U, fi) 
onto {T*V, ri) where fl is the canonical symplectic form on the respective cotangent bundle 
(see Theorem Proposition 6.3.2. in [7]), Theorems 12.11 and 1 2 . 2 1 still hold for a control system 
on any set V that is diffeomorphic to an open subset of R". In this case, we need replace 
r*R" by T*V and use dS]) preferably to (g]) in Theorem O 

2.2 Pontryagin Maximum Principle on Manifolds 

We consider the optimal control problem of finding control u{t) for the control system on 
an n-dimensional manifold M 

± = f{x,u), xeM,ueW (6) 

such that 

ti 

/°(x, is minimized (7) 



with free terminal time ti, and 

x{to) = xq, x{ti) = Xl. (8) 

For convenience, we assume that is a subset of a Euclidean space and that / and are 
smooth. 

Theorem 2.3. Suppose that u{t),to <t<ti is a piecewise continuous optimal control and 
x{t) is the corresponding trajectory. Then, there exists a non-vanishing continuous curve 
(Ao(t), \{t)) e R X r;(j)M such that: 

1. The trajectory {x{t), X{t)) is the flow of 

XHix,X;Xo,u) ^n^dH (9) 
where is the canonical symplectic form on T*M and 

H{x, A, Ao, = Xofix, u) + (A, fix, u)). (10) 

2. Ao < and is constant in t. 

3. u{t) — argmax„gn/ H{x{t), X{t), Xo{t),v) for every t G [to, ti]. 

4. H{x{t),X{t),Xo,u{t)) for every te [to,ti]. 
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Proof. By the Whitney Embedding Theorem [5], we may assume that M is an embedded 
submanifold and a closed subset of for some N G N. By the Tubular Neighborhood 
Theorem 5 , there is an open neighborhood of M in M.-^ with a smooth retraction Try of 
V onto M. Since M is a closed subset of V, by Proposition 2.26 in [5J there is a smooth 
bump function p : — > [0, 1] such that suppp C V, and p{z) — 1 for every z G M. Define 
a control vector field F : V x W ^ TV by 

F{z,u)^ p{z)f{7:v{z),u) eR"". (11) 

It is straightforward to verify that the restriction of F to M agrees with / and that both V 
and M are invariant under the flow of F. Hence, the optimal control problem in ([6]) - (HI 
is equivalent to the problem of finding control u{t), to < t < ti for 

z = F{z,u), zeV,ueW (12) 

such that 



/•ti 

/ /*'(7rv(z), u) (it is minimized (13) 

J to 

with free terminal time ti, and 

z{to) ^ xo e M ^ V, z{ti) £ 7ry\xi) (14) 

where 'iTy^{xi) is an {N — n)-dimensional submanifold of V, and 7ry^(a;i) (IV — {xi}. 

Let u{t), to < t < ti, be a piecewise continuous optimal control for ([6]) - ([8]), and x{t) 
be its optimal trajectory in M. By the equivalence of the two optimal control problems ([6]) 
- ([8]) and (fT2|) - (fT4|) and by the invariance of M, the trajectory 

z{t) := x{t) eM^V (15) 

is the optimal trajectory associated with the control u{t) for - p4|) . By the remark 
below Theorem 12.21 we can apply Theorem 12.21 to (fT^ - (I14p . Hence, there exists a non- 
vanishing continuous curve (poiP(O) £ R x such that with the Hamiltonian H : 
T*V X R X "> R defined by 

H{z,p;po,u) = pof{TTv{z),u) + {p,F{z,u)), 

statements 1-4 in Theorem 12.11 and the transversality condition 

(p(ti),r,,(^^i(xi))) = o (16) 

are satisfied where 

T,,{7Ty\xi))(BT,,M = T,,V ^R^". (17) 
Define a function H :T*M x R x R by 

Hix, A; Ao, u) = Xofix, u) + (A, /(x, u)). (18) 

Let T*V\m be the restriction of T*V to M. Let i : T*V\m ^ T*V be the canonical 
inclusion and tt : T*V\m T*M the canonical projection defined by restricting a £ T*V 
for z G M to TzM. Then, it is easy to show 

Hoi = Hon (19) 

on T*V\m- Since z{t) £ M, {z{t),p{t)) G T*V\m for every t G [^0,^1]- Let us assume 
that both T*V and T*M are equipped with the canonical symplectic forms. We need the 
following lemma: 
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Lemma 2.4. Suppose that M is an n- dimensional embedded submanifold of an N -dimensional 
manifold L. Let T* L\m be the restriction of T*L to M . Let i : T*L\m — > T*L the canonical 
inclusion and n : T*L\m T*M the canonical projection, where both T* L and T* M are 
equipped with the canonical symplectic forms. If H : T*L — > K and i7 : T*M — > R are 
functions such that 

Hoi=HoTT (20) 

on T*L\m, then the Hamiltonian flow ipt with the Hamiltonian H leaves T*L\m invariant, 
so it induces canonically a flow (ft on T* M satisfying t: o ip^ = ip^ o n on T*L\m, and the 
flow ipt is a Hamiltonian flow on T* M for the Hamiltonian H . 

Proof. This lemma is a simple corollary to standard results in geometric mechanics (TJ [7] , 
but we could not find a reference that explicitly states this lemma, so we give a quick proof. 

Let us choose a set of local coordinates {x, y) £ x M^~" for L such that y = 
corresponds to M. Let {{x,y), (a, P)) be the corresponding cotangent bundle coordinates 
for T*L. Then, {x,a,f3) are local coordinates for T*L\m and {x,a) for T*M. In these 
coordinates, (HOI) becomes _ff ((x, 0), (a, /?)) = H{x,a) for every {{x,0), {a, (3)) e T*L\m- 
Then, for any ((x,0), (a,/?)) £ T*L\m, the Hamiltonian vector field Xfj{z) is given by 

i^ = ^((x,0),(a,/3)) = ^(x,a), (21) 

n rr fifi 

r = ^a^,oua,m^—ix,a)-o, 

OH dH 

a^^—^iix,0),{a,(3))^—^{x,a), (22) 

r)H 

$a = ~g^i{x,0),{a,P)). 

Since y'^ = 0, X^{z) is tangent to T*L\m at every z G T*L\m, so its flow ipt leaves T*L\m 
invariant. Then, it is not hard to see that we have a globally well-dcflned flow ipt on T*M 
such that TT o iff ~ iff o t: on T*L\]\j. Equations (|2ip and (j22p imply that ipt is a Hamiltonian 
flow of the Hamiltonian H. □ 

From Lemma [231 with L = y, it follows that {x{t),X{t)) := TT{z{t),p{t)) is a flow of the 
Hamiltonian vector field 

XH{-,Po,u{t)) = n^dH{-;po,u{t)) (23) 

on T*M, where 51 is the canonical symplectic form on T*M . Notice that x{t) is the optimal 
trajectory we began with in ([T5]) and that \{t) is the restriction of p(t) to T^^^t^M , i.e., 

AW^pWIt^^m- (24) 



Setting Aq = po, we see that statement 1 in Theorem 12.31 holds. By statement 2 in The- 
orem 12.11 Aq = po is a non-positive constant, so statement 2 in Theorem 12.31 holds. Since 

{z{t),p{t))eT*V\M, by m 



H{z{t),p{t)-po,v) ^ H{x{t),\{t)-\o,v) (25) 
for every v G W. By statement 3 in Theorem 12. 11 for every t G [to,ti] 

u{t) = argmax^^zw H{z(t),p{t);po,v) 
= avguia^vGW Hix{t),X(t);Xo,v), 
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which imphes that statement 3 in Theorem 12.31 holds. By and statement 4 in Theo- 
rem statement 4 in Theorem l2. 31 holds. 

We now show that the continuous curve (Aq, A(t)) never vanishes. Suppose that there is 
a f e [to, ti] such that (Aq, A(t)) = (0, 0). By Aq being constant and the curve (Aq, A(t)) 
satisfies 

dH 

Ao = 0; A, = -—■,« = 1, ... ,n 

in a local chart containing {x{i), A(i)), which is a (non-autonomous) linear ordinary differ- 
ential equation for (Ao,A) since H in psp is linear in (Ao,A). By the uniqueness of ODE 
solutions, (Aq, A(i)) = (0, 0) as long as {x{t), X{t)) stays in the local chart. By patching up 
a finite number of local charts, we get (Aq, A(<i)) ~ (0,0), which, together with p?|) 
and (|24p . implies that {po,p{ti)) = (0,0), contradicting the non- vanishing assumption on 
{po,p{t)). Therefore, (Aq, A(i)) never vanishes. This completes the proof of Theorem l2.3l □ 

Remark. Notice that using z{ti) G T:y^{xi) in (fT4|) instead of z(ti) = xi is crucial in 
the proof since the condition z{ti) = xi would not guarantee the non- vanishing property of 
(Ao, A(f)) in the last part of the proof. 

Consider the optimal control problem in ([S]) - ([5]) where the fixed endpoint condition in 
dH) is replaced by 

x{to) e ^0, e Si (26) 

where Sq and 5*1 are smooth submanifolds of M. Then, the following theorem holds: 

Theorem 2.5. Let u{t),to < t < ti, be a piecewise continuous optimal control and x{t) 
the corresponding trajectory for Q), ^ and 126\) . Then, it is necessary that there exists a 
nonzero continuous (Aq, A(t)) G M x T*^^^M, which satisfies the conclusions in Theorem \2.S\. 
and, in addition, the transversality conditions 

(Xito), T,ito)So) = 0, (A(ti), r,(t,)^i) = 0. (27) 

Proof. The same arguments in the proof of Theorem 12.31 apply here except that we use 

z{to)eSo^V, Z{tl) e TTy^Sl) 

instead of □ 



Example. Consider a time optimal control problem for the following system on the unit 
2-sphere 5^ that is embedded in R^: 





( » 


Ui 







-Ml 









I 


-U2 






where |ui| < 1, \u2\ < 1, a;(0) = (1,0,0), and x{ti) = (0,0,1). As in the proof of Theo- 
rem l2.3| we can replace the terminal condition x{ti) = (0,0, 1) by 

x{ti) G {a; = (a;^x^x^) G | o;^ = a;^ = 0, < < 2} 

and apply the PMP on R"^. This time optimal control problem is solved from this viewpoint 
in a. 
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